Final Report: The effect of protection of elderly people policy on population-level Covid-19 mortality rate

Abstract

The project aims to investigate the relationship between the strictness level of protection of elderly people policy(PE Policy) and population-level Covid-19 mortality rate(p-mortality rate) from October 1st 2020 to January 31st 2021 for 20 European countries and United Kingdom, and figure out whether it is causal relation based on WHO COVID-19 data and Protection of elderly people policy data. First, descriptive analysis will be conducted to show time series plot of new cases and new deaths in the world and in different WHO regions, and geographic plot of new deaths in the world map over time. Second, in inferential analysis, a three-way ANOVA model will be used to explore relation between the strictness level of PE policy and p-mortality rate and we find that there is a relationship between strictness level of PE policy and p-mortality rate. Also, we use Tukey Kramer method for pairwise comparison and we find that at the significance level of 0.05, the mean p-mortality rate of level-2 PE policy is significantly lower than that of level-1 PE policy. Last, in sensitivity analysis, model assumptions are checked by diagnostic plots.

Introduction

Background and motivation

Nowadays, Covid-19 has become a global problem. To control the outbreak of Covid-19, different governments have taken different measures, including containment and closure polices, economic policies, health system policies and vaccine policies.[1] These policies are released to reduce the number of infectious and deaths caused by Covid-19. However, it remains ambiguous whether some of these policies indeed affect the number of infectious and death cases. Also, some holds that Covid-19 is a kind of epidemic, just like common cold, which cannot be eliminated and what we can do is to reduce the severity of symptoms and mortality. According to WHO, Nearly All Coronavirus Deaths in Europe are people aged 60 and Older.[2] Therefore, my interest is sparked to investigate the relation between the strictness of protection of elderly people policy and Covid-19 mortality rate. Comparing the level of mortality across countries has proven difficult because of inherent limitations in the most commonly cited measures (e.g., case-fatality rates).[3] Also,using case fatality rate is not stable to measure the severity of fatality because cumulative cases keep increasing along the time. Therefore, the project uses population-level mortality rate(i.e., new_death/population*10000). Besides, we also want to figure out which strictness level protection of elderly people policy is associated with the lowest population-level mortality rate. By doing this, we can deeply study the relationship between the policy and Covid-19 mortality rate to provide some suggestions for governments’ policy strictness level. At a certain level, this is also very likely to provide a favorable idea for people to be aware of the effectiveness of certain policy.

Data description

Source

Five data sets are used for this project. The first data set is WHO COVID-19 data updated from 2020-01-03 to 2022-03-01. It contains 183372 observations and 8 variables. The second data set is Protection of elderly people policy data, which is collected by Oxford Covid-19 Government Response Tracker from 2020-01-01 to 2022-02-17. The third data set is population data of each country in the world in 2020. It contains 266 observations and 3 variables. The fourth data set is GDP data of each country in Europe, which contains 37 observations and 2 variables. The fifth data set is elderly ratio data of each country in the world, which contains 266 observations and 3 variables.

Variables

  1. WHO COVID-19 data contains 8 variables:
  • Date_reported: date of the observation reported

  • Country_code: short alphabetic or numeric geographical codes (geocodes) developed to represent countries and dependent areas

  • Country: the country of the observation

  • WHO_region: The World Health Organization (WHO) divides the world into six WHO regions, for the purposes of reporting, analysis and administration.

  • New_cases: the increasing amount of infectious cases in certain country in reported date

  • Cumulative_cases: the cumulative amount of infectious cases in certain country in reported date

  • New_deaths: the increasing amount of death cases in certain country in reported date

  • Cumulative_deaths: the cumulative amount of death cases in certain country in reported date

  1. Protection of elderly people policy data contains 186 rows and 782 columns. Each row represents a country. The first column represents country code, and the second column represents country name. Each of the following columns represents different strictness levels of protection of elderly people policy. Elderly people are defined here relative to the country, region, or territory being coded, for example the local retirement age, and no absolute chronologic age is defined.[4] For countries in Europe and United Kingdom, elderly people are difined to be those persons age 65 and older.
  • 0 - no measures.

  • 1 - Recommended isolation, hygiene, and visitor restriction measures in LTCFs and/or elderly people to stay at home.

  • 2 - Narrow restrictions for isolation, hygiene in LTCFs, some limitations on external visitors and/or restrictions protecting elderly people at home.

  • 3 - Extensive restrictions for isolation and hygiene in LTCFs, all non-essential external visitors prohibited, and/or all elderly people required to stay at home and not leave the home with minimal exceptions, and receive no external visitors.

As can be seen above, from 0 to 3, the restrictions for isolations become narrower, and accordingly the strictness levels of the policy are higher. To combine these two data sets, I plan to rearrange the form of the second data set to make it similar to the form of the first data set.

  1. Population data for each country in the world contains 266 observations and 3 variables. Each observation represents a country in the world. Three variables are:
  • Country: the country of the observation. The variable will be used as the key variable to combine data sets.

  • Country Code: short alphabetic or numeric geographical codes (geocodes) developed to represent countries and dependent areas.

  • Population: the number of people in specific country in Europe.

  1. GDP data for each country in Europe contains 37 observations and 2 variables. Each observation stands for a country in Europe. Two variables are:
  • Country: the country of the observation. The variable will be used as the key variable to combine data sets.

  • GDP(in million euros): the gross domestic product (GDP) in European countries at current market prices in 2020. To get GDP per capita in US dollars, I use GDP/population*1.10 as GDP per capita variable, where 1.10 is the exchange rate.

  1. Elderly ration data for each each country in the world contains 266 observations and 3 variables. Each observation represents a country. Three variables are:
  • Country: the country of the observation. The variable will be used as the key variable to combine data sets.

  • Country Code: short alphabetic or numeric geographical codes (geocodes) developed to represent countries and dependent areas

  • Elderly_ratio: Population percentage of citizens aged 65 years and older in 2020, by country in the world.

Questions of Interest

  1. Is there a relationship between the strictness level of protection of elderly people policy and Covid-19 population-level mortality rate?

  2. Which strictness level protection of elderly people policy is associated with the lowest population-level mortality rate?

Descriptive Analysis

First, after dropping rows with NA and rows with incorrect data which number of cases or deaths are less than zero, a summary table is developed to show range of time, average number of new cases and cumulative cases, average number of new deaths and cumulative cases and average case-mortality rate in different WHO regions.

AFRO (N = 28099) AMRO (N = 26383) EMRO (N = 15591) EURO (N = 37621) SEARO (N = 7016) WPRO (N = 13281)
Avg_New_cases                  
   min 0 0 0 0 0 0
   median 9 146 266 374 325.5 7
   max 37875 1294746 50228 500563 414188 270386
   mean (sd) 269.87 ± 1,288.14 5,439.97 ± 31,090.03 1,294.97 ± 3,183.57 4,480.75 ± 16,985.74 7,888.82 ± 32,357.99 1,444.43 ± 6,371.65
Avg_Cumulative_cases                  
   min 0 0 0 0 0 0
   median 8528 32067 114387 90997 74513 2408
   max 3673257 77129154 7011932 21935509 42924130 4750496
   mean (sd) 75,277.56 ± 311,335.07 1,354,454.05 ± 5,661,715.85 355,763.98 ± 776,103.40 785,902.33 ± 1,896,170.78 2,200,108.62 ± 6,971,664.90 195,745.00 ± 553,880.79
Avg_New_deaths                  
   min 0 0 0 0 0 0
   median 0 3 3 3 3 0
   max 844 8786 709 2291 6148 1290
   mean (sd) 5.74 ± 33.13 97.76 ± 339.45 20.72 ± 53.09 46.78 ± 129.58 108.46 ± 382.73 12.68 ± 45.36
Avg_Cumulative_deaths                  
   min 0 0 0 0 0 0
   median 152 692 1394 1394 841.5 26
   max 99229 916631 135952 346235 513843 54930
   mean (sd) 1,813.53 ± 9,038.13 34,602.73 ± 109,098.96 7,012.09 ± 17,088.93 14,865.20 ± 33,755.73 33,111.92 ± 92,596.06 2,691.03 ± 7,535.38
Avg_case_mortality_rate                  
   min NA NA NA NA NA NA
   median NA NA NA NA NA NA
   max NA NA NA NA NA NA
   mean (sd) 25,392; NaN ± NA 24,153; NaN ± NA 14,421; NaN ± NA 34,329; NaN ± NA 6,590; NaN ± NA 10,780; NaN ± NA
Time_range                  
   min 2020-01-03 2020-01-03 2020-01-03 2020-01-03 2020-01-03 2020-01-03
   max 2022-03-01 2022-03-01 2022-02-28 2022-03-01 2022-02-28 2022-03-01

As we can see above, the data set collects Covid-19 data from 2020-01-03 to 2022-03-01. There are 127989 observations in total. More specifically, EURO region has most observations, which is 37621, while SEARO has least observations, which is 7016. For average new cases, EURO has the largest median and SEARO has the largest mean and standard deviation. For average cumulative cases, EMRO has the largest median and SEARO has the largest mean and standard deviation.For average new deaths, SEARO has the largest mean and standard deviation. For average cumulative deaths, EMRO and EURO have the largest median and AMRO has the largest mean and standard deviation. For case mortatility rate (i.e. new death/cumulative cases*10000), we can see a lot of NA, because when number of cumulative cases equal to zero, case mortatility rate will get NA. Also, number of cumulative cases keeps increasing, even though number of new deaths does not change, case mortality rate will decrease, therefore, it is inappropriate to use case mortality rate in this project to study the relationship. Instead, I use population level mortality rate(i.e., new_death/population*10000).

Next, let us see the trend of new cases and new deaths over time in the world.

First, in the trend plot of new cases over time in the world above, grey lines represent the number of total new cases in the world and different colors lines represent total new cases in different WHO regions. We can find that there are four obvious wave during this time period. The first wave is from October 2020 to Febraury 2021, the second wave is from February 2021 to July 2021, the third wave is from July 2021 to October 2021 and the fourth wave is from October 2021 to now.

Second, from the trend plot of new cases over time in the world below, we can find that there are three obvious wave during this time period. The first wave is from October 2020 to March 2021, the second wave is from March 2021 to July 2021 and the third wave is from July 2021 to October 2021. The time periods of the three waves are corresponding to those of new cases wave. Besides, there are three gentle waves compared the three obvious waves. One is from March 2020 to May 2020, one is from May 2020 to October 2020 and another one is from January 2022 to now.

Then, let us have a view at time series plot of new deaths in different WHO regions. As can be seen below, for EURO and AMRO region, there is obvious rise of new deaths from October 1st 2020 to January 31st 2021. Also, from trend plots before, we can know it is the first obvious wave for both new cases and new deaths, which motivate me to study the relationship of protection of elderly people policy on population-level Covid-19 mortality rate in this time period. Due to data for EURO are more accessible, I plan to study the relationship of the PE policy and population-level mortality rate in EURO from October 1st 2020 to January 31st 2021.

After having an impression on Covid-19 situation all over the world, our attention turns to the status of PE Policy in the world.

In the timeline geographic plot above, the redder the color, the stricter the PE policy. We can find the first country taking out PE policy is Bolivia, and then India and China. We can also see different countries take outa different strict level PE policy. Focus on EURO region from October 1st 2020 to January 31st 2021, we can find from the following plot that PE policy for most countries does not change during this time period. For those countries with policy changes, it is hard to measure the relation/effect of the policy due to lagged effect and different levels of enforcement among different countries. Therefore, it is natural to simplify study procedure by focusing on some countries in which the PE policy does not change during the time.

According to this strategy, to capture most of information about population-level motality rate and avoid unusual fluctuation of population-level mortality rate due to low population, 20 countries in EURO region with population larger than 700,000 and United Kingdom are chosen. Also, due to it is time series data, we plan to aggregate data by week to make sure independence between observations.

In total, the study focuses on twenty one countries: Albania, Belgium, Bosnia and Herzegovina, Croatia, Finland, France, Georgia, Germany, Greece, Greenland, Iceland, Ireland, Israel, Italy, Liechtenstein, Malta, Portugal, Romania, Spain, Sweden and United Kingdom. These countries share two commons: 1. The PE policy did not change from October 1st 2020 to January 31st 2021. 2. Under the first condition, 21 countries have populations in excess of 700,000. Given by our strategy, there is a hidden assumption: the effect of policy has achieved stable condition before October 1st 2020.

Now, let us have a look at the status of PE policy in these twenty one countries. Different columns represent different level of PE policy. From 1 to 3, the strictness level of PE policy increases. Each row represents a country. From the table, we can find Georgia, Greenland, Liechtenstein, Romania, and Spain took out level “1” policy. Belgium, Finland, France, Germany, Iceland, Italy, Malta and Portugal took out level “2” policy. Albania, Bosnia and Herzegovina, Croatia, Greece, Ireland, Israel, Sweden and United Kingdom took out level “3” policy.

TABLE2: The table of status of PE policy among countries
Policy=1 Policy=2 Policy=3 sum
Albania 0 0 123 123
Belgium 0 123 0 123
Bosnia and Herzegovina 0 0 123 123
Croatia 0 0 123 123
Finland 0 123 0 123
France 0 123 0 123
Georgia 123 0 0 123
Germany 0 123 0 123
Greece 0 0 123 123
Greenland 123 0 0 123
Iceland 0 123 0 123
Ireland 0 0 123 123
Israel 0 0 123 123
Italy 0 123 0 123
Liechtenstein 123 0 0 123
Malta 0 123 0 123
Portugal 0 123 0 123
Romania 123 0 0 123
Spain 123 0 0 123
Sweden 0 0 123 123
United Kingdom 0 0 123 123
Sum 615 984 984 2583

Then, how about the population-level mortality(p-mortality) rate in these three countries?

From the interactive time series plot above, we can find the p-mortality for Albania, Belgium, Bosnia and Herzegovina, Croatia, Georgia, Greece and Malta, first increased, and then decreased. For Finland and Iceland, p-mortality was near to 0 and have a slight fluctuation. For France, German and Italy, p-mortality fluctuated between 1 and 2. For Portugal and United Kingdom, p-mortality increased in general. For Romania, Spain, Sweden, Greenland and Israel, p-mortality fluctuated between 0 and 1.For Liechtenstein, p-mortality has a violent fluctuation. In conclusion, there is no common pattern for p-mortality among these eleven countries.

Inferential Analysis

In order to study the relationship between PE policy and p-mortality rate, let us see the boxplot and main effect plot of p-mortality among countries with different policy. From boxplot, there are several outliers in each group. From the main effect plot, mean of p-mortality rate of the third group is the lowest, and mean of p-mortality rate of the first group is the highest. However, the range of p-mortality rate of three groups overlaps, so we can not tell which group has the highest or lowest p-mortality rate just from plots. Under the circumstance, influential analysis is conducted.

We can define a three-way ANOVA model as follows: \[\begin{equation} Y_{ijkt} = \mu_{\cdot\cdot\cdot} + \alpha_i + \beta_j + \gamma_k+ \epsilon_{ijkt}, t = 1,...,n_{ijk}, i = 1,2,3, j = 1,2, k = 1,2 \end{equation}\]

where \(\epsilon_{ijkt}\sim_{i.i.d}N(0, \sigma^2)\) and

\[\begin{equation} \sum^{3}_{1}\alpha_i = 0, \sum^{2}_{1}\beta_j = 0,\sum^{2}_{1}\gamma_k = 0 \end{equation}\] In this model,

  • the index \(i\) represents the level of PE policy: the least strictness PE policy \((i = 1)\), the regular strictness PE policy \((i = 2)\), the most strictness PE policy \((i = 3)\).

The index \(j\) represents elderly ratio: elderly_ratio <= 19.23\((j = 1)\), elderly_ratio > 19.23\((j = 2)\).19.23 is the average elderly ratio among these countries, so we use it as the dividing line. Countries with elderly ratio lower than 20.6 are regarded as low elderly ratio countries in EURO, and Countries with elderly ratio higher than 20.6 are regarded as high elderly ratio countries in EURO.

The index \(k\) represents GDP per capita: GDP per capita <= 37673\((k = 1)\), GDP per capita > 37673\((k = 2)\). 37673 is the averge GDP per capita among these countries, so it is used as dividing line. Similarly. Countries with GDP per capita lower than 37673 are regarded as low GDP per capita countries in EURO, and Countries with GDP per capita higher than 37673 are regarded as high GDP per capita countries in EURO.

  • \(Y_{ijkt}\) is the p-motality rate of the country with the \(i\)th level of PE policy and with the \(j\)th level of elderly ration and with the \(k\)th level of GDP per capita.

  • \(\mu_{\cdot\cdot\cdot}\) indicates the overall p-mortality rate across countries with three types of policy .

  • \(\{\epsilon_{ijkt}\}\) is the error term, which contains unexplained effect on p-mortality.

Then, before checking interaction term, we first check if Box Cox transformation is needed here. A Box Cox transformation is a transformation of non-normal dependent variables into a normal shape and Normality is a necessary assumption for ANOVA model.

For Box Cox transformation, \(\lambda\), which varies from -5 to 5, is the core. The optimal value of \(\lambda\) is selected by optimizing the best approximation of a normal distribution curve.

As above, \(\lambda\) is near zero, so we can do log transformation here. After log transformation of response variable, our three-way ANOVA model becomes: \[\begin{equation} log(Y_{ijkt}) = \mu_{\cdot\cdot\cdot} + \alpha_i + \beta_j + \gamma_k+ \epsilon_{ijkt}, t = 1,...,n_{ijk}, i = 1,2,3, j = 1,2, k = 1,2 \end{equation}\]

where \(\epsilon_{ijkt}\sim_{i.i.d}N(0, \sigma^2)\) and

\[\begin{equation} \sum^{3}_{1}\alpha_i = 0, \sum^{2}_{1}\beta_j = 0,\sum^{2}_{1}\gamma_k = 0 \end{equation}\]

Next, we check if interaction term is needed in this model. Under the circumstance, we set the null and alternative hypothesis as follows and perform likelihood ratio test:

\[\begin{equation} H_0: \{\alpha\beta\}_{ij} =\{\alpha\gamma\}_{ik}= \{\beta\gamma\}_{jk} = 0 \quad v.s. \quad H_1: not \ all \ \{\alpha\beta\}_{ij},\{\alpha\gamma\}_{ik}, \ are \ 0 \end{equation}\]

As we can see the result in the following table: The \(p\) value is 0.6957, which means that we should not reject the null hypothesis at the significance level of 0.05, and therefore the interaction term should be dropped.

#Df LogLik Df Chisq Pr(>Chisq)
1 9 -483.54500764968 NA NA NA
2 6 -484.265934992481 -3 1.44185468560329 0.695753748791215

The assumptions on the proposed additive three-way ANOVA model are:

  1. Factors have a additive and linear influence on the mean of the response variable
  2. Normality: the errors \(\{\epsilon_{ijkt}\}\) are normally distributed.
  3. Independence: the errors \(\{\epsilon_{ijkt}\}\) are mutually independent;
  4. Homogeneity: all the errors \(\{\epsilon_{ijkt}\}\) have the same variance;

Assumptions will be tested in sensitivity analysis, for now, we fit the three-way ANOVA model by using aov() and the detail of the model is shown below:

##              Df Sum Sq Mean Sq F value   Pr(>F)    
## policy        2   10.7   5.367   4.499 0.011833 *  
## Elderly_pop   1    0.2   0.178   0.149 0.699763    
## gdp_per       1   13.2  13.240  11.101 0.000965 ***
## Residuals   318  379.3   1.193                     
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

To answer the primary question of interest, we conduct \(F\) test. The null and alternative hypothesis are as follows:

\[\begin{equation} H_0: \alpha_1 = \alpha_2 = \alpha_3 = 0 \quad v.s. \quad H_1: not \ all \ \alpha_i \ are \ zero \end{equation}\]

We get the result of F test as follows, which are equal to the p values in ANOVA table.

Characteristic p-value
policy 0.012
Elderly_pop 0.7
gdp_per <0.001

The significance level we chose is \(\alpha = 0.05\), the \(p\)-value of policy is \(0.011833\), which indicates that we should reject the null hypothesis conclude that not all \(\alpha_i\) are zero in the significance level of 0.05. Therefore, there is a difference in p-mortality rate across different strictness level of policy.

The \(p\)-value of elferly ratio is \(0.699763\), which indicates that we cannot reject the null hypothesis conclude that not all \(\beta_j\) are zero in the significance level of 0.05.

The \(p\)-value of gdp per capita is \(0.000965\), which indicates that we should reject the null hypothesis conclude that not all \(\gamma_k\) are zero in the significance level of 0.05. Therefore, there is a difference in p-mortality rate across different levels of gdp.

For the second question of interest, we use Tukey-Kramer method at significance level of 0.05.

diff lwr upr p adj
2-1 -0.53066289101899 -0.947848058339654 -0.113477723698327 0.00829465692889853
3-1 -0.373859707309705 -0.773284095456372 0.0255646808369621 0.0720210919992017
3-2 0.156803183709285 -0.161827250967341 0.475433618385912 0.478755973145538

As we can see above, Tukey method suggests that: at the significance level of 0.05, the mean p-mortality rate of level-2 PE policy is significantly lower than that of level-1 PE policy. However, we can not figure out which mean p-mortality rate is lower among level-2 and level-3 policy because p-value is larger than 0.05 and confidence interval of the difference includes 0.

Sensitivity Analysis

We check whether assumptions are plausible by diagnostic plots: From diagnostic plots, we can find:

  • The Residuals v.s. Fitted Values plot shows a nearly equal spread of the residuals along the horizontal axis, therefore the homogeneity assumption seems to hold.

  • The Q-Q plot is still a little left skewed after log transformation, so normality assumption may be violated.

  • The other two plots indicate that no obvious outlier shows a significant influence on the response variable and related factors.

For independence assumption, as we mention before, we aggregate data by week to make sure independence between observations.

Although Q-Q plot is a little left skewed, studies have prove that F-test(ANOVA) is robust to non-normal cases in terms of Type I error[6] and the empirical type II error[7]. Therefore, the result of inferential analysis is feasible given the robust of ANOVA(F-test).

Counclusion

The project uses both descriptive analysis and inferential analysis to explore the relationship between strictness level of protection of elderly policy and population-level mortality rate. The main conclusion for two questions of interests is:

  • There is a relationship between strictness level of PE policy and p-mortality rate

  • At the significance level of 0.05, the mean p-mortality rate of level-2 PE policy is significantly lower than that of level-1 PE policy, but we can not figure which level of strictness corresponding to the lowest p-mortality rate.

Based on our findings, protection of elderly people policy plays an important role in population-level mortality. Therefore, to reduce p-mortality and ensure the freedom of elderly people as much as possible, government should adopt medium level strictness PE policy. That is, narrow restrictions for isolation, hygiene in LTCFs, some limitations on external visitors and/or restrictions protecting elderly people at home.

Admittedly, a caveat of the current analysis is we have no idea about whether the relationship between strictness level of PE policy and p-mortality rate is causal inference or not. In the project, we mainly focus on investigating the relationship, but the relation does not mean adopting medium strictness level policy will cause the decrease of p-mortality rate. To solve the caveat, we need further causal analysis on observational data.

Reference

[1] https://www.bsg.ox.ac.uk/research/research-projects/covid-19-government-response-tracker

[2] https://www.usnews.com/news/world-report/articles/2020-04-02/who-nearly-all-coronavirus-deaths-in-europe-are-people-aged-60-and-older

[3] https://genus.springeropen.com/articles/10.1186/s41118-021-00115-9

[4] https://github.com/OxCGRT/covid-policy-tracker/blob/master/documentation/interpretation_guide.md

[5] https://genus.springeropen.com/articles/10.1186/s41118-021-00115-9

[6] Blanca Mena, María José, et al. “Non-normal data: Is ANOVA still a valid option?.” Psicothema (2017).

[7] Schmider, Emanuel, et al. “Is it really robust?.” Methodology (2010).

[8][9] Mayntz, Renate. Causal mechanism and explanation in social science. No. 20/7. MPIfG Discussion Paper, 2020.

Session info

sessionInfo()
## R version 4.1.2 (2021-11-01)
## Platform: x86_64-apple-darwin17.0 (64-bit)
## Running under: macOS Catalina 10.15.7
## 
## Matrix products: default
## BLAS:   /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRblas.0.dylib
## LAPACK: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRlapack.dylib
## 
## locale:
## [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
## 
## attached base packages:
## [1] stats     graphics  grDevices utils     datasets  methods   base     
## 
## other attached packages:
##  [1] kableExtra_1.3.4     rstatix_0.7.0        lmtest_0.9-39       
##  [4] zoo_1.8-9            gtsummary_1.5.2      car_3.0-12          
##  [7] carData_3.0-5        gplots_3.1.1         viridis_0.6.2       
## [10] viridisLite_0.4.0    hrbrthemes_0.8.0     forecast_8.16       
## [13] qwraps2_0.5.2        anytime_0.3.9        echarts4r.maps_0.0.2
## [16] echarts4r_0.4.3      plotly_4.10.0        forcats_0.5.1       
## [19] stringr_1.4.0        dplyr_1.0.8          purrr_0.3.4         
## [22] readr_2.1.1          tidyr_1.1.4          tibble_3.1.6        
## [25] ggplot2_3.3.5        tidyverse_1.3.1      rmdformats_1.0.3    
## 
## loaded via a namespace (and not attached):
##   [1] colorspace_2.0-2    ellipsis_0.3.2      fs_1.5.2           
##   [4] rstudioapi_0.13     farver_2.1.0        bit64_4.0.5        
##   [7] fansi_1.0.0         lubridate_1.8.0     xml2_1.3.3         
##  [10] extrafont_0.17      knitr_1.37          jsonlite_1.7.2     
##  [13] gt_0.4.0            broom_0.7.11        Rttf2pt1_1.3.10    
##  [16] dbplyr_2.1.1        shiny_1.7.1         compiler_4.1.2     
##  [19] httr_1.4.2          backports_1.4.1     assertthat_0.2.1   
##  [22] fastmap_1.1.0       lazyeval_0.2.2      cli_3.2.0          
##  [25] later_1.3.0         htmltools_0.5.2     tools_4.1.2        
##  [28] gtable_0.3.0        glue_1.6.2          Rcpp_1.0.8         
##  [31] cellranger_1.1.0    jquerylib_0.1.4     fracdiff_1.5-1     
##  [34] vctrs_0.3.8         svglite_2.1.0       urca_1.3-0         
##  [37] nlme_3.1-153        extrafontdb_1.0     broom.helpers_1.6.0
##  [40] crosstalk_1.2.0     timeDate_3043.102   xfun_0.29          
##  [43] rvest_1.0.2         mime_0.12           lifecycle_1.0.1    
##  [46] gtools_3.9.2        scales_1.1.1        vroom_1.5.7        
##  [49] hms_1.1.1           promises_1.2.0.1    parallel_4.1.2     
##  [52] yaml_2.2.1          quantmod_0.4.18     curl_4.3.2         
##  [55] gridExtra_2.3       gdtools_0.2.4       sass_0.4.0         
##  [58] labelled_2.9.0      stringi_1.7.6       highr_0.9          
##  [61] tseries_0.10-49     checkmate_2.0.0     TTR_0.24.3         
##  [64] caTools_1.18.2      commonmark_1.7      bitops_1.0-7       
##  [67] rlang_1.0.2         pkgconfig_2.0.3     systemfonts_1.0.4  
##  [70] evaluate_0.14       lattice_0.20-45     htmlwidgets_1.5.4  
##  [73] labeling_0.4.2      bit_4.0.4           tidyselect_1.1.1   
##  [76] magrittr_2.0.2      bookdown_0.24       R6_2.5.1           
##  [79] generics_0.1.1      DBI_1.1.2           pillar_1.6.4       
##  [82] haven_2.4.3         withr_2.4.3         xts_0.12.1         
##  [85] abind_1.4-5         nnet_7.3-16         modelr_0.1.8       
##  [88] crayon_1.4.2        KernSmooth_2.23-20  utf8_1.2.2         
##  [91] tzdb_0.2.0          rmarkdown_2.11      grid_4.1.2         
##  [94] readxl_1.3.1        data.table_1.14.2   webshot_0.5.2      
##  [97] reprex_2.0.1        digest_0.6.29       xtable_1.8-4       
## [100] httpuv_1.6.5        munsell_0.5.0       bslib_0.3.1        
## [103] quadprog_1.5-8